Structure in the Value Function of Two-Player Zero-Sum Games of Incomplete Information

Decision-making in competitive games with incomplete information is a field with many promising applications for AI, both in games (e.g. poker) and in real-life settings (e.g. security). The most general game-theoretic framework that can be used model such games is the zero-sum Partially Observable Stochastic Game (zs-POSG). While use of this model enables agents to make rational decisions, reasoning about them is challenging: in order to act rationally in a zs-POSG, agents must consider stochastic policies (of which there are infinitely many), and they must take uncertainty about the environment as well as uncertainty about their opponents into account. We aim to make reasoning about this class of models more tractable. We take inspiration from work from the collaborative multi-agent setting, where so-called plan-time sufficient statistics, representing probability distributions over joint sets of private information, have been shown to allow for a reduction from a decentralized model to a centralized one. This leads to increases in scalability, and allows the use of (adapted) solution methods for centralized models in the decentralized setting. We adapt these plan-time sufficient statistics for use in the competitive setting. Not only does this enable reduction from a (decentralized) zs-POSG to a (centralized) Stochastic Game, it turns out that the value function of the zs-POSG, when defined in terms of this new statistic, exhibits a particular concave/convex structure that is similar to the structure found in the collaborative setting. We propose an anytime algorithm that aims to exploit the found structure in order to find bounds on the value function, and evaluate performance of this methods in two domains of our design. As it does not outperform existing solution methods, we analyze its shortcomings, and give possible directions for future research.